Game-theoretical control with continuous action sets

Motivated by the recent applications of game-theoretical learning techniques to the design of distributed control systems, we study a class of control problems that can be formulated as potential games with continuous action sets, and we propose an actor-critic reinforcement learning algorithm that provably converges to equilibrium in this class of problems. The method employed is to analyse the learning process under study through a mean-field dynamical system that evolves in an infinite-dimensional function space (the space of probability distributions over the players' continuous controls). To do so, we extend the theory of finite-dimensional two-timescale stochastic approximation to an infinite-dimensional, Banach space setting, and we prove that the continuous dynamics of the process converge to equilibrium in the case of potential games. These results combine to give a provably-convergent learning algorithm in which players do not need to keep track of the controls selected by the other agents.Comment: 19 page

How Do Adolescents Spell Time Use?

We investigate how household disadvantage affects the time use of 15-18 year-olds using 2003-2006 data from the American Time Use Survey. Applying competing-risk hazard models, we distinguish between the incidence and duration of activities and incorporate the daily time constraint. We find that teens living in disadvantaged households spend less time in non-classroom schooling activities than other teens. Girls spend some of this time in work activities, suggesting they are taking on adult roles. However we find more evidence of substitution into unsupervised activities, suggesting that it may be less structured environments that reduce educational investment.event history models, adolescence, time use

How do Adolescents Spell Time Use?

We investigate how household disadvantage affects the time use of 15-18 year-olds using 2003- 2006 data from the American Time Use Survey. Applying competing-risk hazard models, we distinguish between the incidence and duration of activities and incorporate the daily time constraint. We find that teens living in disadvantaged households spend less time in nonclassroom schooling activities than other teens. Girls spend some of this time in work activities, suggesting they are taking on adult roles. However we find more evidence of substitution into unsupervised activities, suggesting that it may be less structured environments that reduce educational investment.Time use, adolescence, event history models

"Parental Child Care in Single Parent, Cohabiting, and Married Couple Families: Time Diary Evidence from the United States and the United Kingdom"

This study uses time diary data from the 2003 American Time Use Survey and the United Kingdom Time Use Survey 2000 to examine the time that single, cohabiting, and married parents devote to caring for their children. Time spent in market work, in child care as a primary activity, and in child care as a passive activity are jointly modeled using a correlated, censored regression model. Separate estimates are provided by gender, by country, and by weekend/weekday day. We find no evidence that these time allocation decisions differ for cohabiting and married parents, but there is evidence that single persons allocate time differently - as might be expected, given different household time constraints. In the U.S. single fathers spend significantly more time in primary child care on weekdays and substantially less time in passive child care on weekends than their married or cohabiting counterparts, while in the UK single fathers spend significantly more time in passive child care on weekdays. Single fathers in each country report less time at work on weekdays than their married or cohabiting counterparts. In the U.S., single mothers work more than married or cohabiting mothers on weekdays, while single mothers in the United Kingdom work less than married or cohabiting mothers on all days.

Asynchronous Stochastic Approximation with Differential Inclusions

The asymptotic pseudo-trajectory approach to stochastic approximation of Benaim, Hofbauer and Sorin is extended for asynchronous stochastic approximations with a set-valued mean field. The asynchronicity of the process is incorporated into the mean field to produce convergence results which remain similar to those of an equivalent synchronous process. In addition, this allows many of the restrictive assumptions previously associated with asynchronous stochastic approximation to be removed. The framework is extended for a coupled asynchronous stochastic approximation process with set-valued mean fields. Two-timescales arguments are used here in a similar manner to the original work in this area by Borkar. The applicability of this approach is demonstrated through learning in a Markov decision process.Comment: 41 page

Growth or decline in the Church of England during the decade of Evangelism: did the Churchmanship of the Bishop matter?

The Decade of Evangelism occupied the attention of the Church of England throughout the 1990s. The present study employs the statistics routinely published by the Church of England in order to assess two matters: the extent to which these statistics suggest that the 43 individual dioceses finished the decade in a stronger or weaker position than they had entered it and the extent to which, according to these statistics, the performance of dioceses led by bishops shaped in the Evangelical tradition differed from the performance of dioceses led by bishops shaped in the Catholic tradition. The data demonstrated that the majority of dioceses were performing less effectively at the end of the decade than at the beginning, in terms of a range of membership statistics, and that the rate of decline varied considerably from one diocese to another. The only exception to the trend was provided by the diocese of London, which experienced some growth. The data also demonstrated that little depended on the churchmanship of the diocesan bishop in shaping diocesan outcomes on the performance indicators employed in the study

STUDIES ON THE HETEROLOGOUS IMMUNOGENICITY OF A METHANOL-INSOLUBLE FRACTION OF ATTENUATED TUBERCLE BACILLI (BCG) : II. PROTECTION AGAINST TUMOR ISOGRAFTS

A methanol-insoluble residue (MER) of phenol-killed attenuated tubercle bacilli (BCG), which has been reported previously to be capable of evoking heightened resistance to infection with antigenically unrelated microorganisms, was found to affect as well the resistance of highly inbred mice against tumor isografts. In most instances, the MER evoked heightened resistance against the tumor implants, but heightened susceptibility was the effect induced against two of the tumors tested, and no effect was elicited against one neoplasm. It is suggested that the heightened susceptibility occasionally produced by pretreatment with MER may also be of immunological nature, i.e. immunological enhancement. Treatment with MER was more effective when administered some time before tumor challenge than when given simultaneously with, or after, tumor implantation. The protective effects manifested against some tumors were of a high order, a significant number of animals rejecting the neoplastic implants, and were displayed even when several months elapsed between treatment and challenge. Living BCG and intact phenol-killed bacilli also evoked heightened resistance against some of the tumors tested, and in one experiment living BCG proved effective whereas MER did not. On the whole, however, MER was the most active (and least toxic, as shown previously) of the several tubercle bacillus preparations tested. MER elicited heightened reactivity against first transplant generation tumors as well as against tumors maintained for considerable periods of time by repeated animal passage, and against spontaneously arising as well as against induced neoplasms. The experimental parameters necessary to demonstrate maximal effects varied somewhat from tumor to tumor. In general, however, single intraperitoneal injections of small quantities of MER, of the order of 0.25 to 1.0 mg, afforded the best protection

BOSH:Bayesian Optimization by Sampling Hierarchically

Deployments of Bayesian Optimization (BO) for functions with stochastic evaluations, such as parameter tuning via cross validation and simulation optimization, typically optimize an average of a fixed set of noisy realizations of the objective function. However, disregarding the true objective function in this manner finds a high-precision optimum of the wrong function. To solve this problem, we propose Bayesian Optimization by Sampling Hierarchically (BOSH), a novel BO routine pairing a hierarchical Gaussian process with an information-theoretic framework to generate a growing pool of realizations as the optimization progresses. We demonstrate that BOSH provides more efficient and higher-precision optimization than standard BO across synthetic benchmarks, simulation optimization, reinforcement learning and hyper-parameter tuning tasks

Robustness Properties in Fictitious-Play-Type Algorithms

Fictitious play (FP) is a canonical game-theoretic learning algorithm which has been deployed extensively in decentralized control scenarios. However standard treatments of FP, and of many other game-theoretic models, assume rather idealistic conditions which rarely hold in realistic control scenarios. This paper considers a broad class of best response learning algorithms, that we refer to as FP-type algorithms. In such an algorithm, given some (possibly limited) information about the history of actions, each individual forecasts the future play and chooses a (myopic) best action given their forecast. We provide a unifed analysis of the behavior of FP-type algorithms under an important class of perturbations, thus demonstrating robustness to deviations from the idealistic operating conditions that have been previously assumed. This robustness result is then used to derive convergence results for two control-relevant relaxations of standard game-theoretic applications: distributed (network-based) implementation without full observability and asynchronous deployment (including in continuous time). In each case the results follow as a direct consequence of the main robustness result